The present invention relates to systems and methods for streaming and archiving video with geographic anchoring of frame contents.
It is known to provide geographical information relating to the location at which still or video images were sampled, and in some cases, to incorporate the location information as metadata within the image or video data. This technology is commonly referred to as “go-tagging”, and allows the resulting media to be associated with corresponding locations on a map. Examples of this technology may be found in U.S. Pat. Nos. 6,282,362, 6,504,571, 6,741,790 and 7,042,470.
The location data for the aforementioned geo-tagging of video is typically derived from sensors associated with the imaging device, such as GPS sensors, inertial sensors and/or inclination sensors. Such sensors provide various levels of precision as to the location of the imaging device, and possibly also the direction in which the imaging device is facing, but do not provide accurate information about the geographical coverage of the video content. Thus, for example, if a user of a database wishes to view video of a specific geographical location, the user may need to sift through many videos taken from locations which theoretically could have viewed the location of interest including videos which were taken from nearby locations but not facing in the correct direction.
Even if geo-tag data were supplemented with information about the pointing direction of the imaging device, the actual content of the images is very sensitive to angular precision. Even a small angular error in the pointing direction can result in a major inaccuracy in the geographic area which is visible in the images. This is particularly true in situations where narrow field-of-view optics is used to view distant objects, as is common in the field of military reconnaissance and surveillance.
In military applications, it is known to employ real-time scene matching technology to correlate information in viewed images with images from a geographic database, thereby determining the geographic coordinates of each point viewed in a sampled image. In other words, each pixel of each image is effectively mapped to a corresponding geographic location. This allows reliable identification of points of interest and transfer of geographically-linked information between platforms viewing a region from different viewing directions. The scene matching processing is computationally heavy, and requires access to an extensive infrastructure of reference images. As such, it cannot readily be repeated on demand for re-use of the video in other locations or at a later time.
Military systems are today highly network based, and it may be advantageous to make video sampled from a remote sensor available at various different locations in a network. In most cases however, due to bandwidth and processing capacity limitations, it is not feasible to provide the functionality of geographic anchoring of the images at multiple locations. Specifically, the scene matching processing used to achieve geographic anchoring of the scene contents typically requires that the original full-resolution image data, uncompressed or compressed only by “lossless” techniques, be available to ensure reliable results. The bandwidth available in most networks, and particularly in wide area networks (WAN), typically prohibits transfer of such uncompressed or minimally compressed data. After compression by high compression ratio lossy techniques for network transmission, the data is typically insufficient to perform reliable scene matching for geographic anchoring.
There is also a need to provide for storage and retrieval of video sampled from multiple sources according to the geographical coverage of the video, The above-mentioned U.S. Pat. No. 6,504,571 discloses a system and method for querying a digital image database based on various parameters including geographic location data. Here again, since the indexing of the video is based on the location of the imaging device, there is a wide margin of uncertainty regarding the geographical content of each image, leading to location mismatches between a requested location and coverage of video sequences retrieved from a database. The user typically has no way of knowing whether a particular location of interest is included within the video or not without checking all videos recorded within a certain range of the location of interest, or at least, a large subset of such videos which were taken in the right general direction. In the case of a large database, such a task may be impractical
There is therefore a need for a system and method for facilitating transmission and use of video sequences which would include precise geographical anchoring of the video frame content. It would also be advantageous to provide a system and method for storage and retrieval of video on demand based upon accurate indexing of the geographical coverage of the content of the video frames.